A Chaotic Dynamical System that Paints

Can a dynamical system paint masterpieces such as Da Vinci’s Mona Lisa or Monet’s Water Lilies? Moreover, can this dynamical system be chaotic in the sense that although the trajectories are sensitive to initial conditions, the same painting is created every time? Setting aside the creative aspect of painting a picture, in this work, we develop a novel algorithm to reproduce paintings and photographs. Combining ideas from ergodic theory and control theory, we construct a chaotic dynamical system with predetermined statistical properties. If one makes the spatial distribution of colors in the picture the target distribution, akin to a human, the algorithm first captures large scale features and then goes on to refine small scale features. Beyond reproducing paintings, this approach is expected to have a wide variety of applications such as uncertainty quantification, sampling for efficient inference in scalable machine learning for big data, and developing effective strategies for search and rescue. In particular, our preliminary studies demonstrate that this algorithm provides significant acceleration and higher accuracy than competing methods for Markov Chain Monte Carlo (MCMC). Is it possible to design a dynamical system that paints like a human? Given the availability of efficient modern printing technologies, this may seem like a trivial problem. However, the manner in which a modern printer prints is fundamentally different when compared to a human painting a picture. Roughly speaking, a printer scans each pixel in a pre-determined order and, using a color palette, deposits the appropriate amount of ink for each pixel. In comparison, given the same color palette, a human paints by first capturing the high level (or large scale) features of the picture and then goes on to fill in the low level (or detailed) features. In this paper, we are not attempting to model or mimic 1 ar X iv :1 50 4. 02 01 0v 1 [ nl in .C D ] 8 A pr 2 01 5 the intelligence and creativity of humans when perceiving and painting pictures. Rather, our objective is to design an algorithm that reproduces the human actions of painting by first capturing the large scale features followed by small scale details. Our algorithm is based on the construction of a deterministic (no randomness or noise) dynamical system (described by a set of governing differential equations), which visits states with frequencies prescribed by a user defined distribution. Our approach is related to the theory of ergodicity. Ergodic systems are dynamical systems with the property that time averages of functions along trajectories are equal to spatial averages; the associated statistical distributions are known as invariant measures of the system [1]. In this work, we construct an ergodic dynamical system where one can prescribe the statistical distributions of the underlying dynamics. By prescribing the color distributions as target distributions for the ergodic system, individual trajectories for each color (the empirical distributions) converge to the desired invariant distributions, thus tracing out the original painting or picture. The underlying dynamical system is chaotic in the sense that it exhibits sensitivity to initial conditions (as shown in the supplementary material, the system has three Lyapunov exponents > 0), but nonetheless leads to robust recreation of the picture irrespective of initial condition. Note that positive Lyapunov exponents are the primary characteristic of chaos [2, 3]. This algorithm can potentially be used to drive a robot that reproduces paintings/pictures. The ramifications of this approach extend beyond the applicability of designing robotic systems that can paint [4, 5]. The challenging task of efficient sampling of complex probability distributions lies at the heart of a wide range of problems. For example, sampling probability distributions is one of the most important tasks in statistical inference and machine learning, and is typically achieved by Markov Chain Monte Carlo (MCMC) methods [6, 7, 8, 9, 10]. Variational methods [11, 12] are a popular alternative for statistical inference that rely on the construction of bounds on the likelihood function that may not always be tight [13]. In this work, we restrict ourselves to MCMC based sampling approaches. MCMC methods are often plagued by slow mixing [14], particularly when distributions are complex and multi-modal. We believe that our approach presents an exciting alternative for sampling complex distributions for Bayesian inference and machine learning in the big data setting. In the supplementary material accompanying this manuscript, we present comparisons of Metropolis-Hastings [15], Hamiltonian MCMC [16, 17], and slice sampling [18] with chaotic sampling. We find that our ap-